6

The Nature of Information

51

bits, this being the average number of questions admitting yes/no answers required

to specify the length to the nearest inch, as the reader may verify. On the other

hand, were we to have some prior information, according to which we had good

reason to suppose the length to be close to 9 inches (perhaps we had previously

requested the wood to be chopped to that length), the probabilities of the lengths 8,

9, and 10 inches would perhaps be 0.25 each, and the sum of all the others would

be 0.25. The existence of this prior knowledge would somewhat reduce the quantity

of information gained from the measurement, namely tothree fourths log Subscript 2 Baseline 4 plus one fourth log Subscript 2 Baseline 36 equals 2.79 ³

4 ^{log2 4 + 1}

4 ^{log2 36 = 2.79}

bits. Should the ruler have been marked off in tenths of an inch, the measurement

would have yielded considerably more information, namely log Subscript 2 Baseline 120 equals 6.91log2 120 = 6.91 bits,

again assuming all the probabilities of the wood being any particular length to be

equal (i.e., StartFraction 1 Over 120 EndFraction ¹

120 ^each).

Variety. One of the most striking characteristics of the natural, especially the living,

world around us is its variety. This variety stands in great contrast to the world

studied by the methods of physics and chemistry, in which every electron and every

proton (etc.) in the universe are presumed to be identical, and we have no evidence to

gainsay this presumption. Similarly, every atom of helium (Superscript 4⁴He) is similar to every

other one, and indeed it is often emphasized that chemistry could only make progress

as a quantitative science after the realization that pure substances were necessary for

the investigation of reactions and the like, such that a sample of naphthalene in a

laboratory in Germany would behave in precisely the same way as one in Japan. ²

If we are shown a tray containing balls of three colours, red (r), blue (b), and white

(w), we might reasonably assert that the variety is three. Hence, one way to quantify

variety is simply to count the number of different kinds of objects. Thus, the variety

of either of the sets StartSet normal r comma normal b comma normal w EndSet{r, b, w} and StartSet normal r comma normal b comma normal b comma normal r comma normal w comma normal r comma normal w comma normal w comma normal b EndSet{r, b, b, r, w, r, w, w, b} is equal to three; the set

StartSet normal r comma normal r comma normal w comma normal w comma normal w EndSet{r, r, w, w, w} has a variety of only two, and so forth. The objects considered should

of course be in the same category; that is, if the category were specified as “ball”, then

we would have difficulty if the tray also included a banana and an ashtray. However,

one could then redefine the category.

If there were only one kind of ball, say red, then our counting procedure would

yield a variety of one. It is more natural, however, to say that there is no variety if all

the objects are the same, suggesting that the logarithm of the number of objects is a

more reasonable way to quantify variety. If all the objects are the same, the variety is

then zero. We are, of course, at liberty to choose any base for the logarithm; if the base

is 2, then conventionally the variety is given in units of bits, a contraction of binary

digit. Hence, two kinds of objects have a variety of log Subscript 2 Baseline 2 equals 1log2 2 = 1 bit, and three kinds

givelog Subscript 2 Baseline 3 equals StartFraction log Subscript 10 Baseline 3 Over log Subscript 10 Baseline 2 EndFraction equals StartFraction 0.477 Over 0.301 EndFraction equals 1.58log2 3 = ^{log10 3}

log10 2 ^{= 0.477}

0.301 ^{= 1.58 bits. The variety in bits is the average number of}

2 This is only an approximation. In classical physics, presumably immutable laws predetermine only

the general character of phenomena; a continuous variety of realizations is admitted, and specific

features are determined by the initial conditions (cf. Duns Scotus’ haecceitas). This is in contrast

to the quantum realm, in which individual atoms have well-defined specific qualities, and their

identities are immutable, independent of their previous histories (Ramsden 2010).